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(54) Statletlcal database correction of alphanumeric account numbers for speach recognition 
and touch-tone recognition 



(57) A method and apparatus for recognizing an 
identifier entered by a user. A caller enters a predeter- 
mined identifier through a voice input device (15) or a 
touch-tone keypad of a telephone handset. A signal rep- 
resenting the entered identifier is transmitted to a re- 
mote recognizer (30), which responds to the identifier 
signal by producing a recognized output intended to 
match the entered identifier. The present invention com- 
pares this recognized identifier with a list (35) of valid 
reference identifiers to determine which one of these ref- 
erence identifiers most likely matches the entered iden- 



tifier. In performing this determination, the present in- 
vention employs a confusion matrix (45), which is an ar- 
rangement of prolsabilities that indicate the likelihood 
that a given character in a particular character position 
of the reference identifier woukJ be recognized by the 
recognizer as a character in the corresponding charac- 
ter position of the recognized identifier. This determina- 
tion yields an identifier recognition probability for every 
reference Identifier, and the present invention selects 
the reference identifier with the highest identifier recog- 
nition probability as most likely corresponding to the en- 
tered identifier. 
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Description 

Background of the Invention 

The present invention relates to a method and ap- 
paratus for recognizing an identifier that is entered into 
a system by a user, and in particular, to a method and 
apparatus that recognizes such an Input Identifier on the 
basis of a probability determination that selects, from 
among a plurality of predetermined reference identifiers, 
a reference identifier that has the highest probability of 
matching the input identifier. 

Most institutions, such as banks and department 
stores, allow customers to access over the telephone a 
wide variety of services and account information. Before 
the advent of touch -tone telephones, a customer would 
obtain these sen/ices and information through interact- 
ing with a live operator. As touch-tone telephones be- 
came more prevalent In homes, these institutions began 
switching to automated customer-access systems. After 
dialing a telephone number, a customer using such sys- 
tems would be asked to enter an account number or 
identifier. As used herein, the terms 'account number' 
and 'identifier" are used interchangeably, and they refer 
to a string of characters that may comprise a plurality of 
letters, numbers, or bc^h. Furthermore, as used herein, 
an identifier may be used not only to identify a user, but 
also may be used as an identifier for identifying a par- 
ticular product or service offered by an institution. In the 
first generation of automated customer-access sys- 
tems, a user would enter such an identifier by sequen- 
tially pressing a series of keys provided on the telephone 
keypad. Each pressed key would correspond to a differ- 
ent character in the identifier. The pressing of these keys 
woukj produce a series of tones that would be provided 
over a telephone network to the institution. At the insti- 
tution, the series of tones would be decoded to produce 
the entered Identifier, and if the identifier entered by the 
user was determined to correspond to a valid identifier, 
then the user would be allowed to enter commands, 
again through the telephone keypad, that would provide 
access to whatever services would be offered by the in- 
stitution. 

The next generation of automated customer-ac- 
cess systems eliminates the use of telephone keypads 
to verify the identity of a valid user. Instead of entering 
an identifier through a telephone keypad, a user would 
be prompted to speak the identifier into the telephone 
handset. For example, the user may speak into the tel- 
ephone the identifier "JB123E". The user's voice signal 
would be transmitted over the phone lines to the finan- 
cial institution, which would employ a speech recogni- 
tion system to produce a recognized identifier that is in- 
tended to correspond exactly to the identifier spoken by 
the user. 

Nevertheless, such exact correspondence Is quite 
difficult to attain, mosWy due to the deterioration of voice 
signals that routinely occurs over conventbnal tele- 



phone lines. In particular, as a voice signal is transmitted 
to a remote location, conventional telephone lines intro- 
duce into such signals noise and restrictive band limita- 
tions. Such a deterioration present in a voice signal may 

5 cause a remote speech recognizer to produce a recog- 
nized output that does not correspond to the spoken 
identifier. Because of the limitations introduced into the 
voice signal by the telephone lines, the speech recog- 
nizer may confuse similar sounding letters and num- 

10 bers. Thus, a speech recognizer may confuse the letter 
"A" with the number "8", the letter "K", or the letter "JV 
Similarly, the speech recognizer may confuse the letter 
"C with the letter "D" or the number '3\ For example, 
given that a user speaks the identifier ■JB123E' into a 

IS telephone, the speech recognizer may produce 
"AE123D' as an output Accordingly, a need exists to 
enhance the accuracy of such speech recognition sys- 
tems and overcome the limitations introduced into voice 
signals by typical communication lines, such as, for ex- 

20 ample, conventional telephone lines. 

Similarly, touch -tone recognition systems also mis- 
takenly recognize the wrong identifier. Accordingly, a 
need also exists to enhance the accuracy of such touch - 
tone recognition systems. 

2S 

Summary of the Invention 

In order to overcomethese deficiencies, the present 
invention is directed to a method and 8^>paratus that en- 

30 hances the reliability of a system intended to recognize 
multi-character identifiers provided by a remote user. 

In a first representative embodiment of the present 
invention, the remote user enters an Identifier by speak- 
ing into a voice input device. A recognized voice output, 

55 representing the predetermined identifier, is provided to 
a processor. This recognized identifier is based on an 
klentifier spoken Into a telephone by a user. The proc- 
essor Is coupled to a database that contains a plurality 
of valid identifiers. These identifiers residing in the da- 

40 tabase are referred to as reference identifiers. The proc- 
essor Is also coupled to a mennory that stores a plurality 
of probabilities arranged as at least one confusion ma- 
trix. The processor obtains from the database a refer- 
ence identifier for comparison to the recognized identi- 

45 fier. Starting at the first character position for both the 
recognized kJentifrer and the reference identifier, the 
processor uses the confusion matrix to obtain the prob- 
ability that the character found in the first character po- 
sition of the reference Identifier would be recognized as 

so the character found in the first character position of the 
recognized identifier. Probabilities are obtained in such 
a manner for every character position in the reference 
and recognized identifiers. Thus, after obtaining the 
probability for the characters of the reference and rec- 

ss ognized identifiers found in their respective first charac- 
ter positions, the processor obtains the probability that 
the character found in the second character position ol 
the reference identifier would be recognized as the char- 
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acter found in the second character position ot the rec- 
ognized identifier, etc.. AUer all the probabilities have 
been obtained with respect to the recognized identifier 
and the reference identifier, an identifier recognition 
probability is determined based on the obtained proba- 
bilities. For example, the identifier recognition probabil- 
ity may be based on the multiplication of siich obtained 
probabilities. The processor repeats this procedure for 
every reference identifier stored in the database. After 
this procedure is performed for every reference identifi- 
er, the processor selects from the plurality of reference 
identifiers the reference identifier most likely matching 
the spoken identifier. This selection may be accom- 
plished, for example, by selecting the reference identi- 
fier corresponding to the highest identifier recognition 
probability; this selected identifier is presented to the us- 
er as the one most likely matching the spoken identifier. 

In another representative embodiment, the user is 
provided with an opportunity to indicate whether the se- 
lected reference identifier corresponds to the spoken 
identifier. If the user indicates that a match exists, then 
the user is allowed to access the semces provided by 
the institution that implements the present invention. If, 
however, the user indicates that the selected reference 
identifier does not match the spoken identifier, the 
present invention provides a new plurality of reference 
Klentrfiers from the database. This new plurality of ref- 
erence identifiers does not include the previously select- 
ed reference Identifier. Recognition probabilities are cal- 
culated and assigned to every reference identifier in this 
new plurality, In accordance with the procedure de- 
scribed above, and the reference identifier in this new 
plurality corresponding to the highest probability is se- 
lected as being the most likely to match the spoken iden- 
tifier If such a match is indicated by the user, the user 
is given access to the account corresponding to the 
identifier. If the user Indcates again that a match does 
not exist, then the processor repeats the same proce- 
dure with yet another new plurality of reference identifi- 
ers. This latest plurality of reference identifiers rein- 
states the previously excluded reference identifier, but 
eliminates from consideration the moGt recently select- 
ed reference identifier. 

According to another representative embodiment of 
the present invention, no reference identifier may be se- 
lected as a possible match for the spoken identifier un- 
less the identifier recognition probability corresponding 
to the reference identifier exceeds a predetermined 
threshold. In this embodiment, after each reference 
identifier under consideration is assigned a correspond- 
ing identifier recognition probability, the processor de- 
termines which, if any, reference identifiers are associ- 
ated with identifier recognition probabilities that exceed 
a predetermined threshokl. If no such reference identi- 
fiers are currently available, the processor discards the 
results of the procedure and prompts the user to speak 
the identifier again. If any identifier exceeds the prede- 
termined threshokl, the processor selects the highest 



. from among these probabilities. This selected reference 
identifier is presented to the user for a" ' 
whether it Indeed matches the spoken identifier. IT tne 
user indicates that no such match exists, the P^®^^J 
s invention can either re-prompt the ^^^^ ^P®^ 
kJentifier again, or present to the user the reference 
Wentifier corresponding to the next highest probability 
that exceeded the predetermined threshold. So long as 
the list of reference identifiers with probabilities in ex- 
10 cess of the threshold is not exhausted, the processor 
continuously presents to the user the reference identifier 
with the next highest probability until a positive nnatcn is 

Indicated. , ,. 

According to yet another representative embodi- 
es ment. the present invention is impien^^nted in a touch 
tone recognition system. In this embodimerit, a user en- 
ters an kientifler through a conventional keypad of a 
touch-tone telephone. The system produces a recog- 
nized identifier and establishes an identifier recognition 
20 probability for every reference Identifier in rnenriory. m 
the same manner as discussed above for the sp^cn 
recognition system. The touch-tone recognition system 
would then selects the identifier mo^t likely matching the 
Input identifier from these reference identifiers. P-or ex- 
2S ample, the kJentifier most likely matching ^^^^ input iden- 
tifier may correspond to the reference identifier with the 
highest klentifier recognition probaf'^^y 
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Brief Description of the Drawlnag 

aher features and advantage ^®.^'"^T\'rH 
vention will become apparent from tne following detailed 
descrlptton, together with the drav^f^Qs. m wnicn. 

Fig. 1 shows a block diagram ^ speech recogni- 
tion system in accordance wit^ the present mven- 

Shows a confusion matri:< for arranginga plu- 
lamy of probabilrties indicative o1 the likelihood tl^t 
a particular character in a reference identifier was 

spoken by a user; . .... 

Fig. 3 Shows a first predetermined identrfier gram- 

'^'4 shows a second precJetermined identifier 



grammar; 

Fig. 5 shows a flow diagram ccp 



_ rres ponding to a first 
eirtj^lmJJil torctete^irtng'vs/hidi reference iden- 
tifier was nnost likcely spoken « . 
Fig. 6 shows a flow diagram co r responding to a sec- 
ond embodiment for determining vvhich reference 
identifier was most likely spolc^" by a user. 
Fig. 7 shows a flow diagram CO r responding to a third 
embodiment for determining v^hich reference iden- 
tifier was nr»8t lil<ely spoken ^ ^ . ^ 
Fig. 8 Shows a block diagram oi a '^"^^^-"'"f^^- 
ognition system in accordance with the present .n- 
vention; and . . 

Figs. 9(a) and 9(b) illustrate confus«n matnces 
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used in conjunction with the system of Fig. 8. 
Detailed Description of the Invention 

Fig. 1 illustrates a system that rmpienrients the 
speech recognition routine of the present Invention. The 
system of Fig. 1 is merely an example of one kind of 
system that is capable of supporting the present speech 
recognition routine, and it should be appreciated that the 
present speech recognition routine is compatible with 
numerous other applicalions. 

The system 10 of Fig. 1 includes a voice input/out- 
put device 15, which may comprise a conventional tel- 
ephone or microphone. A user wishing to access a par- 
ticular service provided by the system would be prompt- 
ed to speak into voice input/output device 15 a prede- 
termined identifier For purposes of this discussion, the 
user shall be assumed to speak a valid identifier into 
device 10. This voice signal can be carried over a tele- 
phone line to a public telephone network interface 65, 
which interfaces the voice signal generated by voice in- 
put/output device 10 to the remaining components of the 
speech recognition system. Of course, any wired or 
wireless connection coukJ convey the voice signal to the 
speech recognition system. The system of Fig. 1 further 
includes an A/D converter 20, which converts the analog 
voice signal provided by interface 65 into a digital signal. 
A/D converter 20 supplies the digitized voice signal to 
speech recognizer 30, which may comprise, for exam- 
ple, a HARK 3.0 recognizer, which is manufactured by 
BBN Co. . After employing a recognition routine, for ex- 
ample, the Hidden Markov Model, speech recognizer 30 
provides as an output a recognized identifier, which may 
or may not correspond to the dentifier that the user 
spoke into the telephone. The recognized output is pro- 
vided to an input of CPU 40. CPU 40 is configured to 
determine whether the recognized identifier corre- 
sponds to any one of a plurality of valid identifiers stored 
in database 35. which may comprise a hard disk or any 
other suitable storage medium capable of storing a large 
number of account numbers. 

The identifier recognition routine that CPU 40 uses 
to verify the validity of a recognized identifier is stored 
in identifier verification module 75. Used in conjunction 
with the routine of module 75 Is a confusion matrix, 
which is stored in memory 45 and which shall be de- 
scribed along with the routine of module 75 soon here- 
after. CPU 40 controls a voice prompt device 60, which 
may comprise DIALOGIC telephone interface cards. 
CPU 40 causes prompt device 60 to issue voice inquir- 
ies to a user at voice input/output device 15. For exam- 
ple, the voice prompt device 60 may issue an inquiry 
such as 'Please tell me your identifier'. The system of 
Fig. 1 also includes a data input devk:e 50, such as a 
keyboard, a CD-ROM drive, or a floppy drive, and the 
system of Fig. 1 is also provided with a display 55. 

Fig. 2 illustrates a confusbn matrix that Is used by 
CPU 40 to validate the veracity of a recognized identifier 



provided by speech recognizer 30. For the ®^P'® P^" 
vided in Fig. 2. the identifier grammar shall be LLNNiMt. 
That is. each one of the valid identifiers s^^®^ m data- 
base 35 is six characters long in which the first twochar- 

5 actor positions may comprise only letters of the alpha- 
bet, the third through fifth character positions may com- 
prise only numerals 0-9. and the last character position 
may comprise either a letter or a numeral. »n order to 
avoid confusing the letter "O" with the numeral 0 . the 

10 identifier grammar may be configured to exclude the tet- 
ter -O- as a possible letter to be used in the first, secorxJ, 
or last character positions, and recognizer 30 would be 
configured to recognize the numeral 'O" when rt is spo- 
ken by a user either as "oh" or 'zero'. Of course, the 

IS characters that constitute the identifier grammar can be 
configured to be of whatever length and may compnse 
any combination of letters, numerals, or both. 

Since illustrating a confusion matrix for the en ire 
alphabet is not necessary to explain the operation of the 

20 identifier recognition routine, the confusion matrix or Mg. 
2 is limited to a portion of the alphabet. Furthermore, the 
matrb( is not provided with every entry because the fol- 
lowing discussion shall refer to a limited number of loen- 
tifiersand the entries provided in the matrix correspond 

25 to those letters that are included in this limited group of 
identifier. Thus, all the blanks in Fig- 2 should be con- 
sidered to be zero. Of course, when the system of the 
present invention is implemented, the confusion ma roc 
wouldbe provided with acomplete ©etof entries and the 

30 database would be provided with a large annount of iden- 
tifiers, tor example, 100,000. ^ 

The confusion matrix of Fig. 2 is readasfoMows ttie 
vertical columns correspond to letters and nurnbers tr^ 
were recognized by recognizer SO. and the honzontal 

55 rows correspond to letters and numbers spoken into the 
telephone. Of course, the confusion matrix of Fig 2 may 
be configured In reverse, in which the horizontal rows 
correspond to letters and numbers recognized by rec- 
ognizer 30. and in which the vertical columns corre- 

40 spond to letters and numbers spoken into the telephone. 
• The decimals that are provided in the confusion matrix 
represent different probabilities. For example, based on 
the confusion matrix of Fig. 2. given that "A is recog- 
nized by recognizer 30. the proba^Hrty that A wasspo- 

45 ken by a user into a telephone is 50%. For the recog- 
nized letter -A", there is also a probability o^ 30% ha 
•J" was spoken, and there is a probability of 20% that 
"8" was spoken when "A" was recognized. 

The particular probabilities that are provided in Fig. 

50 2 are determined in advance through experimentation^ 
and they are tailored to suit the particular recognizer 30 
that is used in the system of Fig., t Thus, when a par- 
ticular recognizer is to be used in the system of Fig. 1. 
a test group of persons repetitively provides P^onuncia- 

55 tions of each of the letters and n u merals. and the rec- 
ognized output of recognizer 30 for each pronunciation 
isrecorded. In oider that the results of these tests incor- 
porate the influence of the noise ^nd bandwidth limita- 
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tlonsthat affect speech recognizers operating under re- 
al conditions when receiving voice signals from a tele- 
phone line, the vocal pronunciations of this test group 
of persons may be provided to the speech recognizer 
over a telephone line. From these various 'trial runs" of s 
the recognizer 30, the probabilities that characterize the 
recognition accuracy of the recognizer are established, 
and these probabilities can be entered as a confusion 
matrix into memory 45 through data input device 50. 
Since different recognizers exhibit different recognition io 
accuracies, if recognizer 30 is to be replaced with a dif- 
ferent recognizer, than a confusion matrix correspond- 
ing to the replacement recognizer must be entered into 
memory 45. Or alternatively, memory 45 may store in 
advance a plurality of predetermined confusion matrices is 
corresponding to different recognizers, so that when a 
replacement recognizer is implemented, the corre- 
sponding confusion matrix may be selected by entering 
a command through input device 50. It should be noted 
that the probabilities of a confusion matrix- need not be 20 
arranged in matrix form, but may be arranged as an ar- 
ray, or as any other data structure capable of associating 
a recognized and spoken character in temns of a prob- 
ability. 

Another alternative embodiment is shown in Fig. 3, 25 
in which a separate confusion matrix Is provided for 
each portion of the code. For example, if the predeter- 
mined identifier grammar described above is used, then 
CPU 40 would access from memory 45 (1 ) a "letters on- 
ly' confusion matrix when analyzing the first two char- 00 
acter positions (2) a "numbers only" confusion matrix for 
the next three character positions, and a "numbers and 
letters' confusion matrix, such as the one in Fig. 2. for 
the last character position. By using such separate con- 
fusion matrices, the recognition accuracy of the overall 
system is improved. For example, when analyzing a 'let- 
ters only' character position, the "letters only" confusion 
matrix will not allow for the possibility that a recognized 
■A" was mistaken by recognizer 30 for a spoken "8". 
Thus, the probability that a spoken "A" is actually rec- 40 
ognized as an "A" increases because one possibly con- 
fusing character, the number ■8", has been eliminated 
from consideration. 

Of course, Fig. 3 does not illustrate the only possible 
correspondence between confusion matrices and char- 45 
acter positions in a character string. As Fig. 4 illustrates, 
each character position in a predetermined grammar 
may be assigned its own confusion matrix. Furthermore, 
each particular character position may be constrained 
as to the number of possible characters that can occupy so 
that positton. For example, in Fig. 4, the only characters 
that can occupy the first character position are the letters 
"A" through 'M'; the only characters that can occupy the 
second character position are the letters 'N" through 
■2'; the only characters that can occupy the third char- ss 
acter position are the numbers '0" through "3"; the only 
characters that can occupy the fourth character position 
are the numbers "4" throu^ "6"; the only characters that 
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can occupy the fifth character poeiti«>'^ ^jon would not 
•7" through "9"; and the last character P^^^^ permissibly 
be limited as to which letters or nunnb©r 
occupy that last position. the identifier 

When the predetermined grarnnn ^.^^^p js assigned 
is limited as in Fig. 4, each ch aracter P^^^ particular con- 
a confusion matrix that is matched ^^^^racter position, 
stralnts placed on the corres por^ding ponds lo a con- 
Thus, the first character position ^^^^^q possibility that 
fusion nnatrix thai accounts only ^<^^ oken; the second 
letlersAthroughM could have been sP^^^^^jgjon matrix 
character position corres poncds that letters N 

that accounts only for the possib'^^^^ ^^^^ character 
through Z could have been spoKeri; ^^inx that ac- 
position corresponds lo a cont^sio ^^^^ „q. ^j^^ough 
counts only for the possibility that nu^^ position corre- 
-3- were spoken; the fourth charac^^^^^g Q^iy for the 
sponds to a confusion matrix that ^^^^ spol<en; 

possibility that numbers "4- throug^^^^ to a contusion 
the fifth character position oorresp<=> .^.jj^y^hatlhenum- 
matrlxthataccounts only for the P^^^ sixth charac- 
bers '7' through "9'' were spoKen ; ^trixthat takes 
ter position corresponds to sl oonf ^ 
into account both letters ancd rium^^ . pigs. 3and4may 

The various contusion nnatrices ^py 40 may 

be stored in advance In nne mory .^pending on which 
automatically switch among ^hern,^^^ ^jch character 
predetermined grammar is us^^ ^ 
positk>n is being currently ana ly^^^^^t rate the operation 

The flow charts of Figs. 5-"7 '"^^^ identifier ventica- 
of CPU 40 in accordance with* ^^''^fLpending on the par- 
tionrcutinesstoredinmodul© 7^- "-^^^tect among these 
ticular implementation, CPU 4-0 m^V ^,3506 on acom- 
stored routines either autonnatic^^y^^vice 50. With re- 
mand entered through data inp^^ go, under the control 
spectlo Fig. 5, voice prompt d g\/'^^^ ^, y^\s identifier (step 
ofCPU40, prompts the user to ^^^^0 spoken identifier 
100). For illustrative purpose^ « it is assumed to be 
shall be assumed to be AE1 ^ to this identifier is 

valid. The voice signal corres pon^n ^^pverterfiO. which 
supplied through interface 6 S to ^^rresponding to the 
generates a digitized voice sig"^^ signal is suppl»©a 
spoken identifier. This digiti;Z©cJ digital code (recog- 

to recognizer 30, which produc^^ include each char- 
nized identifier) that may or m^V ^j^en by the user (step 
aclor of the identifier that wss -^^ed identifier shali be 
110). In this example, the rescso^'l' ^^en creates a list 0! 
assumed to be JB1 23E. C F=>LJ ^^xxf iers stored in data- 
reference Identifiers from •<=^^^plained later, this list 
base 35 (step 1 20). As sha 1 1 t> ^.^^ntitiar stored in data- 
may or may not Include eve ry »^ 



base 35. 



After generating the list identifier in the list 

CPU 40 goes to the first r&i&r^^^ ^j^st character in the 
(step 130). and in particular, tc=> }l^^rs {step ^^Y^'^^^^l 
recognized and reference icS&r-i^*\' firet identifier in tne 
sake of simplicity, assume X^^^ ^ins ^r*^"^ "'^'^^^ 
list is AE1 23D. CPU 40 th en o 



reference 



identifiers, 
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confusion matrix. Given that J, the first character in the 
recognized identifier, was recognized. CPU 40 deter- 
mines from the confusion matrix the probability that A 
was spoken (step 1 50). As explained before, the confu- 
sion matrix used may be one that encompasses letters 
or numbers or both, or one that encompasses a subset 
of letters or numbers. 

After obtaining such a probability. CPU 40 deter- 
mines whether all the character positions of the refer- 
ence and recognized identifiers have been analyzed 
(step 160). If not, CPU 40 moves to the next character 
position for both the recognized and reference identifi- 
ers (step 1 65) and consults the confusion matrix again 
to determine the probability of recognizing the particular 
character of the recognized identifier when the corre- 
sponding character of the reference identifier was spo- 
ken. This procedure is carried out until a probability is 
determined for every character position. After the last 
character positions have been analyzed, CPU 40 multi- 
plies ail the obtained probabilities for each character po- 
sition (step 170). The result of this multiplication repre- 
sents, given that the recognized identifier was recog- 
nized, the probability that the reference identifier used 
in this iteration was actually spoken. This probability is 
referred to as an identifier recognition probability. Of 
course, the identifier recognition probability may be de- 
termined in ways other than multiplication of the confu- 
sion matrix probabilities. The present invention is 
deenned to encompass alternative methods of determin- 
ing identifier recognition probabilities that are based on 
confusion matrix probabilities. In the example given 
above, given that JB123E Is recognized, the probability 
that AE123E is spoken may be determined to be 30%. 
based on the confusion matrix. After determining this 
probability, CPU 40 then goes to the next reference 
kdentifier (step 165], and repeats steps 140-170 to ob- 
tain another probability. This process is repeated until 
an Identifier recognition probability has been deter- 
mined for every reference Identifier In the list. 

Once all these probabilities have been determined, 
CPU 40 determines the reference identifier that has 
been assigned the highest probability (step 190). CPU 
40 then causes voice prompt device 60 to ask the user 
if the reference identifier with the highest probability 
matches the identifier originally spoken by the user 
(Step 200). If a match exists* then the speech recogni- 
tion procedure ends and the user accesses the informa- 
tion or services to which his is entitled. It should be ap- 
preciated that selecting the reference identifier with the 
highest probability is not the only way to select the ref- 
erence identifier that most likely matches the spoken 
identifier. The present invention is intended to encom- 
pass other techniques relying on probabilistic determi- 
nations to select such a reference identifier. 

If a match does not exist, then CPU 40 creates a 
new list of reference identifiers that excludes the most 
recently determined identifier with the highest probabil- 
ity (step 215). In step 215, CPU 40 also reinstates any 



prevfously determined 'highest probability" identifiers. 
For example, if the first iteration of the procedure in Fig. 
5 produces a mismatch based on reference identifier 
JD875C. then JD875C is excluded from being ccan- 

5 pared to a recognized identifier during a second iteration 
of the procedure of Fig. 5. If the second iteration pro- 
duces a misnr»tch based on reference identifier 
BC421J, then in the third Iteration, BC421J is excluded 
from consideration and JD875C is reinstated into the list 

10 of reference identifier. This reinstatement is done in 
case a user actually indicated in a previous iteration that 
a match did exist between the reinstated idenlifier and 
the spoken identifier, but the systenn erroneously under- 
stood this indication to mean that a mismatch existed. 

IS Thus, if a user indicated that a match existed based on 
kjentifier JD875C. but the system nnisunderstood the us- 
. er to mean that a mismatch existed, identifier JD875C 
would be reconsidered again because of the reinstate- 
ment in step 21 5. Of course, this reconsideratton would 

20 not occur until after another complete iteration is done 
without the identifier JD875C. That is, the user would 
indicate that the highest probability identifier in the next 
iteration is not the correct one, and the system would 
reinstate identifier JD875C in a subsequent iteration. 

25 Thus, the system would be provided with another op- 
portunity to Identify identifier JD876C as the one that the 
user spoke into the telephone. 

Once a new list is created (step 215). CPU 40 re- 
prompts the user to pronounce the identifier again. After 

30 speech recognizer 30 produces a recognized identifier, 
the procedure above for detennining the highest prob- 
ability identifier is repeated. The re-prompting step and 
the step of producing a second recognized identifier is 
optional, since the originally spoken identifier and rec- 

36 ognized identifier can be compared to the new list 

In another embodiment, the entire result of an iter- 
ation is discarded If all the calculated probabilities are 
below a certain threshold. As iltust rated In Fig. 6, a user 
is prompted for an Identifier (step 300). After producing 

40 a recognized identifier corresponding to the spoken 
identifier (step 310). CPU 40 determines an identifier 
recognition probability for each reference identifier in the 
same manner as illustrated in Fig. 5 (step 320). After 
determining a probability for every reference identifier, 

45 CPU 40 determines if any of these probabilities exceeds 
a predetermined threshold (step 330). The threshold is 
determined experinr>entally and is set low enough so that 
any probability falling betow or matching the threshold 
would not be reasonably regarded as corresponding to 

50 a con-ectly recognized Identifier If probability ex- 
ceeds the predetermined threshokd. then the procedure 
returns to step 30O. If at least one probability exceeds 
the threshold, then the identifier corresponding to the 
highest among these probabilities is presented to the 

55 user (steps 340. 350). If the user iridicates that a match 
exists (step 360). then the procedure ends (step 370). 
If no match exists, then the proc^ciure begins again at 
step 300. 



6 



11 



EP 0 848 536 A2 



12 



Alternatively, as Indicated in Fig. 7, CPU 40 can be 
configured to create a list of tliose reference identifiers, 
each of these reference identifiers corresponding to an 
identifier recognition probability that exceeds a prede- 
termined threshold (step 440). The reference identifier 5 
in this list that corresponds to the highest identifier rec- 
ognition probability Is presented to the user (step 450). 
If the user indicates that a match does not exist (step 
460), then instead of reprompting the user, CPU 40 
presents the user with the identifiercorrespondingtothe io 
next highest probability in the list (step 490). Such a pro- 
cedure could be repeated until the user indicated that a 
match existed (step 460) or until CPU 40 exhausted all 
the probabilities that exceeded the predetermined 
threshold (step 480). is 

In yet another embodiment of the present invention, 
Fig. 6 illustrates a touch-tone recognition system 20 that 
also implements at least one confusion matrix in order 
to enhance the recognition accuracy of the system. The 
components in Figs. 1 and 8 that are referred to by the 20 
same reference character can be considered to be the 
same component. A user enters through touch-tone in- 
put device 11 5 an identifier that may comprise an alpha- 
numeric string of characters. Touch-tone input device 
115 may comprise the keypad of a typical touch-tone 25 
telephone. In this embodiment It shall be assumed, for 
exemplary purposes only, the key numbered "2" on the 
keypad con-esponds to letters "A". "B", and "C"; that the 
key numbered "3* corresponds to letters 'D', "E", and 
"F"; that the key numbered "4* corresponds to letters 30 
•G". 'H", and "1"; that the key numbered "5" corresponds 
to letters 'J". "K", and 'L"; that the key numbered "S" 
corresponds to letters "M". "N". and 'O*; that the key 
numbered "7" corresponds to letters "P", "Q", "R", and 
■S"; that the key numbered "8" corresponds to letters 
"T', "U". and "V; and that the key numbered "9' corre- 
sponds to letters "W", "X", "Y", and "Z". Of course, this 
particular correspondence between numerteal keys and 
alphabetical letters is only one of many possible ar- 
rangements that may be established in touch-tone rec- 40 
ognrtion systems. 

Fig. 9(a) illustrates one possible confusion matrix 
that may be stored in confusion matrix memory 45 of 
system 20. The matrix shows several blocks of non-zero 
probabilities. For example, assuming that the intended 
character is a letter, pressing the key numbered "2" sig- 
nifies the letters "A", "B", or "C". Since pressing this key 
may signify arry of these letters, each letter has a 33% 
chance that it has been recognized correctly by recog- 
nizer 130. For keys "7" and "S'. each of which corre- so 
spends to 4 letters, each of these letter has a the prob- 
ability of 25% that it has been recognized correctly. 

Fig. 9(b) shows the confusion matrix for the numeric 
characters. Since each key on the keypad is assigned 
to only one number, there is a 100% chance that the ss 
touch-tone recognition system 20 will correctly recog- 
nize the number that was input Into the system by the 
user. 



When the system of Fig. 8 is in operation, a user 
wouldenter an identifier through touch-tone input device 
115. The identifier in this system may be defined accord- 
ing to the same grammar as the Identifiers used in the 
system of Fig. 1 . Touch-tone recognizer 130 would sup- 
ply to processor 40 a string of alphanumeric character 
that purports to match the one supplied by the user. 
Processor 40 would verily the accuracy of this recog- 
nized identifier in the same way as the system of Fig^l : 
by calculating for each reference identifier stored in da- 
tabase 35 a particular identifier recognition probability 
based on the recognized Identifier and selecting from 
these reference identifier the Identifier most likely 
matching the input identifier. The selection of the closest 
reference identifier may be based, for example, on the 
reference identifier corresponding to the highest identi- 
fier recognition probability Furthermore, the procedures 
outlined in Figs. 5-7 are capable of being implemented 
in the system of Fig. 8. the only difference being that the 
system of Fig. 8 requires a user to enter an identifier 
through a touch-tone telephone keVP^^- 

The speech recognition systenn and toudi-tone r^ 
ognition system of the present invention are not limited 
to accessing account information over a telephone, but 
instead may be applied to a wide variety of environ- 
ments that require the validatton of a user's identity or 
that requirea user to identify a particular feature or serv- 
tee associated with the system- P^r example, the 
present invention may be implemented in a secunty sys- 
tem intended to limit access to a facility to only those 
persons that speak a valid identifier into a microphone 
installed at an emrance of the faci m Another possible 
implementation for the present invention is at a remote 
point-of-sale terminal or a remote money-access sta- 
tion, in which a customer's PIN n umber would be en- 
tered vocally through a microphone Furthemnore, each 
of the Items stored In database 35 may be configured to 
Identify a particular product or sen/i -^e that a user desi res 
to purchase. Thus, In this configuration, a user calling a 
catalog retailer may identify each product that is to be 
purchased by a particular alphanumeric, numeric, or al- 
phabetical code associated with th^tproduct. Aftera us- 
er enters such a product identification code into the teN 
ephone. the system of Fig. 1 would perform the identifier 
recognition described above to cor rectly identify the de- 
sired product or service. 



Claims 

1 . Am ethod of recognizing an icJentlfier entered by a 
user, the Identifier including a- first plurality of pre- 
determined characters, the mi^thod comprising the 
steps of: 

a) providing a recognizee:^ Identifier based on 
the entered identifier, the recognized identifier 
comprising a second plurality predetermined 
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characters; 

b) providing a plurality of reference identifiers, 
each one of the plurality of reference identifiers 
comprising a different plurality of predeter- 
mined characters; 

c) obtaining, for each character position in at 
least one of the reference Identifere and each 
character position in the recognized identifier, 
a probability that a character in the at least one 
reference identifier is recognized as a character 
found in the corresponding character position 
in the recognized identifier; 

d) determining an identifier recognition proba- 
bility based on the obtained probabilities; 

e) repeating steps c) and d) for every reference 
identifier in the plurality of reference identifiers, 
each one of the plurality of reference identifiers 
being associated with a corresponding identifi- 
er recognition probability; and 

. f) selecting the reference identifier most likely 
matching the entered identifier based on the 
plurality of obtained recognition probabilities. 

2. The method according to claim 1 , wherein each one 
of the entered identifier, the recognized Identifier, 
and the plurality of reference identifiers comprises 
a plurality of alphanumeric characters. 

3. The method according to claim 1 , wherein each one 
of the entered identifier^ the recognized identifier, 
and the plurality of reference Identifiers comprises 
a plurality of numbers. 

4. The method according to claim 1 , wherein each one 
of the entered identifier, the recognized identifier, 
and the plurality of reference identifiers comprises 
a plurality of alphabetical letters. 

5. The method according to claim 1 . wherein the ob- 
tained probabilities are arranged as at least one 
confusbn matrix. 

6. The method according to claim 5, wherein the at 
least one confusion matrix Includes at least one 
confusion matrix corresponding to alphabetical let- 
ters only, at least one confusion matrix correspond- 
ing to numbers only, and at least one confusion ma- 
trix corresponding to a combination of alphabetical 
letters and numbers. 

7. The method of claim 1 , wherein the entered identi- 
fier is entered by the user spealcing the identifier into 
a voice input device. 

8. The method according to claim 7. wherein the rec- 
ognized identifier is provided by a speech recogniz- 
er. 



9. The method according to claim 1 . wherein the ref- 
erence identifier selected in step 0 corresponds to 
the highest identifier recognition probability. 

5 10. The method of claim 1 . wherein the entered identi- 
fier is entered by the user through a touch-tone input 
device. 

11. The method of claim 10, wherein the recognized 
10 identifier is provided by a touch-lone recognizer. 

12. The method according to claim 1 . further compris- 
ing the steps of: 

IS g) prompting the user to indicate whether the 

selected reference identifier matches the en- 
tered identifier; 

h) if the user indicates that the selected refer- 
ence identifier matches the entered identifier, 

20 acknowledging the user as having entered a 

valid identifier; 

i) if the user indicates that the selected refer- 
ence identifier does not nnatch the entered iden- 
tifier 

25 

]) providing a second plurality of reference 
identifiers, the second plurality of reference 
identifiers including every reference identi- 
fier except the selected reference identifi- 

30 er; 

k) repeating steps c) and d) for eveiy ref- 
erence Identifier included in the second 
plurality of reference Identifiers, each one 
of the reference identifiers of the second 

36 plurality of reference identifiers being as- 

sociated with a second corresponding rec- 
ognition probability; and 
1) selecting from the second plurality of ref- 
erence identifiers the reference Identifier 

40 most likely matching the entered identifier 

based on the second corresponding iden- 
tifier recognition probabilities. 

13. The method according to claim 12. wherein the ref- 
45 erence identifier selected in st ep 1 ) corresponds to 

the highest identifier recognition probability of the 
second corresponding identifier recognition proba- 
bilities. 

50 14. The method of claim 12. further comprising the 
steps of: 

m) prompting the user to indicate whether the 
reference identifier selected in step 1 ) matches 
55 the entered Identifier; 

n) if the user Indicates that the, reference iden- 
tifier selected in step 1) r-natches the entered 
kJentifier, acknowledging tne useras havingen- 
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tered a valid identifier; 

o) it the user indicates that the reference Iden- 
tifier selected in step 1 ) does not match the ref- 
erence identifier selected in step 1 ]: 

p) reinstating the reference identifier se- 
lected In step f) Into a third plurality of ref- 
erence Identifiers, the third plurality of ref- 
erence identifiers further including every 
other reference identifier except for the ref- 
erence identifier selected in step 1); 
q) repeating steps c) and d) for every ref- 
erence identifier included in the third plu- 
rality of reference identifiers, every refer- 
ence identifier in the third plurality of refer- 
ence identifiers being associated with a 
new corresporx^ing identifier recognition 
probability; and 

r) selecting from the third plurality of refer- 
ence identifiers the reference identifier 
most likely matching the entered identifier 
based on the new corresponding identifier 
recognition probabilities. 

16. The method according to claim 14, wherein the ref- 
erence identifier selected in step r) corresponds to 
the highest recognition probability of the new corre- 
sponding identifier recognition probabilities. 

16. The method of claim 1 . wherein the obtained prob- 
abilities are obtained from an external storage me- 
dium. 

17. The method of claim 1, wherein the obtained prob- 
abilities are obtained from a data input device. 

18. The method of claim 1 , wherein the obtained prob- 
abilities are obtained from a memory device. 

19. The method of claim 1 , wherein before the selecting 
step f). the method further comprises the step of de- 
termining asubset of the plurality of reference iden- 
tifiers, each of the reference identifiers included in 
the subset corresponding to an Identifier recogni- 
tion probability higher than a predetennined thresh- 
old, wherein the reference identifier selected in step 
f) is selected from the subset of the plurality ofref- 
erence identifiers. 

20. The method of claim 19. further comprising the 
steps of: 

g) prompting the user to indicate whether the 
reference identifier selected in step f) matches 
the entered Identifier; 

h) If the user indicates that the selected refer- 
ence Identifier matches the entered identifier, 
acknowledgingthat the user has entereda valid 



identifier; , rof*»r- 

i) if the user indicates that 
ence identifier does not match theentered rien 

tifier 

j) selecting the reference identifier corre- 
sponding I the next higf^^^t 'dentifler rec^ 
ognition probability within the subset of the 
plurality of reference idenHners, 
r. u- -r to ndcate whether 

k) prompting the user to^ .^ j) 

the reference identifier 
corresponds to the ente^d 
1) if the user indicates that *e reference 
identiflerselectedlnstepH'^atc'iesme en- 
tered identifier, acknowledging that the us 
er has entered a valid ide" 'le^^ 
m) if «he user indicataj Jhat ^^.^ 
identifier selected m step J> " 

the entered identifier. ^^P^^^^^^.f 
through 1) in accondance with the re er 
ence id Jifiers included 'n the subset of 
the plurality of reference identifiers. 

. - on identifier entered 

21. An apparatus for recognizing ' ^. «««trtiii 

by a user, the entered identifier 
i^Hy Of predetemiined characters, the system com 

prising: 

. • = recognized identifier 

means for recemng a r« J j^^d 
based on the entered idenj^^^^'*^ ^ 
Identifier comprising a sec^"° P'""^ " 
determined characters; 

a f„st memoo. that stores a p.uraj^^^^^^ 

ence identifiers, ^^^h or> 
identifiers comprising a differenipiu. j 

rs^rd;™^storasap.u^;^^^^^^^^ 

abimies. each Of the ^ro^^^^^Z l^!. 
a prcbabiltty that a cena,n^^^ ^^^^^^^^^ 
ter corresponds to a cert^^ « 
a processor, in comryunic^t ^^^^ 
for receiving, the first mer^«'V. 
memoor. and determining. «^ a correspond- 
plurality of reference 'der--J''"®'J ^^^^ 

ing identifier '-'''^j;^^?2r''r^'^->- P-'^^""" 
corresponding identifier '^^ ^t the plu- 

ties being determined on 
rality of probabiWies store^J 

ory. the processor selecti '^f ^l^,^,^ identifier 
tifier most likely matching the en^e .^.^^ 

based on the Identifier recs'^snmon h 

. ^ I aim 21, wherein the 

22. The apparatus according to '^'^ to the 
selected reference identifier T" 

ss highest identifier recognition l="°''*'""^' 

_ . itfaim 21, wherein each 

23. Theapparatusaccordingtocl^'r" „ 
one of the entered identifier, t recognize 
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Her. and the plurality of reference identifiers com- 
prises a plurality of alphanumeric characters. 

24. The apparatus according to claim 21 , wherein each 
one of the entered identifier, the recognized identi- s 
fier, and the plurality of reference identifiers com- 
prises a plurality of numbers. 

25. The apparatus according to claim 21 , wherein each 
one of the entered identifier, the recognized identi- io 
fier, and the plurality of reference identifiers com- 
prises a plurality of alphabetical letters. 

26. The apparatus according to claim 21. wherein the 
plurality of probabilities are arranged in the second '5 
memory as al least one confusion matrix. 

27. The apparatus according to claim 26, further com- 
prising a selecting means, in communication with 

the processor, for selecting among the at least one 20 
confusbn matrix, 

28. The apparatus according to claim 26, wherein the 
at least one confusion matrix includes at least one 
confusion matrix corresponding to alphabetical let- 2S 
ters only, at least one confusion matrix correspond- 
ing to numbers only, and at least one confusion ma- 
trix corresponding to a combinatbn of alphabetical 
letters and numbers. 
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FIG. 2 
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FIG, 6 
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FIG. 7 
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FIG. 10(a) 

Aft rnrr/^u i t i^i uunDno cthuu/v vt 
AO \/LfLrun 1 J^LM^lU^UK o \ UYWX TZ 


A 

A 


.J J. J J. J J 


n 
b 


IX 17. IT 


c 


77 77 77 


D 


.33.33.33 


E 


.33.33.33 


F 


J3.33.33 


G 


.33.33.33 


H 


.33.33.33 


I 


.33.33.33 


J 


.35.33.33 


K 


.33.33.33 


L 


.33.33.33 


M 


.33.33.33 


N 


.33.33.33 


0 


.33.33.33 


P 


.25i5 .25.25 


Q 


.25.25.25.25 


R 


.25.25.25.25 


S 


.25.25.25.25 


T 


•33 •33*33 


U 


.33.33.33 


V 


.33.33.33 


w 


.25.25.25.25 


X 


.25.25.25.25 


Y 


.25.25.25.25 


z 


.25.25.25.25 



19 



EP 0 848 536 A2 



FIG. 10(b) 
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